15 research outputs found
Learning Models For Corrupted Multi-Dimensional Data: Fundamental Limits And Algorithms
Developing machine learning models for unstructured multi-dimensional datasets such as datasets with unreliable labels and noisy multi-dimensional signals with or without missing information have becoming a central necessity. We are not always fortunate enough to get noise-free datasets for developing classification and representation models. Though there is a number of techniques available to deal with noisy datasets, these methods do not exploit the multi-dimensional structures of the signals, which could be used to improve the overall classification and representation performance of the model.
In this thesis, we develop a Kronecker-structure (K-S) subspace model that exploits the multi-dimensional structure of the signal. First, we study the classification performance of K-S subspace models in two asymptotic regimes when the signal dimensions go to infinity and when the noise power tends to zero. We characterize the misclassification probability in terms of diversity order and we drive an exact expression for the diversity order. We further derive a tighter bound on misclassification probability in terms of pairwise geometry of the subspaces. The proposed scheme is optimal in most of the signal dimension regimes except in one regime where the signal dimension is less than twice the subspace dimension, however, hitting such a signal dimension regime is very rare in practice. We empirically show that the classification performance of K-S subspace models agrees with the diversity order analysis. We also develop an algorithm, Kronecker- Structured Learning of Discriminative Dictionaries (K-SLD2), for fast and compact K-S subspace learning for better classification and representation of multidimensional signals. We show that the K-SLD2 algorithm balances compact signal representation and good classification performance on synthetic and real-world datasets. Next, we develop a scheme to detect whether a given multi-dimensional signal with missing information lies on a given K-S subspace. We find that under some mild incoherence conditions we must observe ��(��1 log ��1) number of rows and ��(��2 log ��2) number of columns in order to detect the K-S subspace.
In order to account for unreliable labels in datasets we present Nonlinear, Noise- aware, Quasiclustering (NNAQC), a method for learning deep convolutional networks from datasets corrupted by unknown label noise. We append a nonlinear noise model to a standard convolutional network, which is learned in tandem with the parameters of the network. Further, we train the network using a loss function that encourages the clustering of training images. We argue that the non-linear noise model, while not rigorous as a probabilistic model, results in a more effective denoising operator during backpropagation. We evaluate the performance of NNAQC on artificially injected label noise to MNIST, CIFAR-10, CIFAR-100, and ImageNet datasets and on a large-scale Clothing1M dataset with inherent label noise. We show that on all these datasets, NNAQC provides significantly improved classification performance over the state of the art and is robust to the amount of label noise and the training samples
Beyond Labels: Empowering Human Annotators with Natural Language Explanations through a Novel Active-Learning Architecture
Real-world domain experts (e.g., doctors) rarely annotate only a decision
label in their day-to-day workflow without providing explanations. Yet,
existing low-resource learning techniques, such as Active Learning (AL), that
aim to support human annotators mostly focus on the label while neglecting the
natural language explanation of a data point. This work proposes a novel AL
architecture to support experts' real-world need for label and explanation
annotations in low-resource scenarios. Our AL architecture leverages an
explanation-generation model to produce explanations guided by human
explanations, a prediction model that utilizes generated explanations toward
prediction faithfully, and a novel data diversity-based AL sampling strategy
that benefits from the explanation annotations. Automated and human evaluations
demonstrate the effectiveness of incorporating explanations into AL sampling
and the improved human annotation efficiency and trustworthiness with our AL
architecture. Additional ablation studies illustrate the potential of our AL
architecture for transfer learning, generalizability, and integration with
large language models (LLMs). While LLMs exhibit exceptional
explanation-generation capabilities for relatively simple tasks, their
effectiveness in complex real-world tasks warrants further in-depth study.Comment: Accepted to EMNLP 2023 Finding
Effective object tracking in unstructured crowd scenes
by Ishan Jindal and Shanmuganathan Rama
Semantic description of a video using representative frames
by Ishan Jindal And Shanmuganathan Rama